Constructing Imperfect Recall Abstractions to Solve Large Extensive-Form Games

Extensive-form games are an important model of finite sequential interaction between players. The size of the extensive-form representation is, however, often prohibitive and it is the most common cause preventing deployment of game-theoretic solution concepts to real-world scenarios. The state-of-the-art approach to solve this issue is the information abstraction methodology. The information abstraction reduces the size of the original large extensive-form game by removing information available to players; hence merging the information sets which from their decision points. Since the players have to play identical strategy in the merged information sets, the size of the strategy representation in the abstracted game can be significantly smaller than in the original game. The abstracted game is then solved, and the small resulting strategies are used in the original game. The majority of existing information abstraction approaches create abstracted games where players remember all their actions and all the information they obtained in the abstracted game – a property denoted as a perfect recall. Remembering all the actions, however, causes the number of decision points of the player (and hence also the size of his strategy) to grow exponentially with the number of actions taken in the past. On the other hand, relaxing the perfect recall requirement (resulting in so-called imperfect recall abstractions) can significantly increase the computational complexity of solving the resulting abstracted game. Hence, there is only a limited amount of work that focuses on using imperfect recall abstractions. These approaches either use computationally complex algorithms to solve the abstracted game or restrict to trivial subclasses of imperfect recall abstractions that are easy to solve. In this work, we take a novel approach to imperfect recall information abstractions, which does not require any specific structure of the imperfect recall abstracted game nor does it use computationally complex algorithms to solve it. Instead, we introduce two domain-independent algorithms FPIRA and CFR+IRA which are able to start with an arbitrary imperfect recall abstraction of the solved ∗Corresponding author Email addresses: [email protected] (Jǐŕı Čermák), [email protected] (Viliam Lisý), [email protected] (Branislav Bošanský) Preprint submitted to Artificial Intelligence March 15, 2018 ar X iv :1 80 3. 05 39 2v 1 [ cs .G T ] 1 4 M ar 2 01 8 two-player zero-sum perfect recall extensive-form game. The algorithms simultaneously solve the abstracted game, detect the missing information causing problems and return it to the players. This process is repeated until provable convergence to the desired approximation of the Nash equilibrium of the original game. We experimentally demonstrate that even when the algorithms start with trivial coarse imperfect recall abstraction, they are capable of approximating Nash equilibrium of large extensive-form games using abstraction with as little as 0.9% of information sets of the original game. Moreover, the results suggest that the relative memory requirements of the algorithms will further decrease as the size of the solved game increases.